1. Field of the Invention
The present invention relates to image processing apparatuses and methods, programs, and recording media. More particularly, the invention relates to an image processing apparatus and method, a program, and a recording medium in which recoding can be performed by using information concerning coding previously performed on corresponding data.
2. Description of the Related Art
In a system for transmitting moving picture signals to remote places, such as in a videoconference system or a videophone system, image signals are coded by being compressed according to line correlation or inter-frame correlation in order to efficiently utilize transmission channels.
When compressing an image signal, the image signal is coded so that a bit stream to be generated has a predetermined bit rate. In practical use, however, because of the situation of the transmission channel, it may be necessary to change the bit rate of the bit stream.
When editing a transmitted image signal in a broadcasting station, editing is performed in units of frames, and thus, information concerning each frame of the image signal should be independent of information concerning the other frames of the image signal. A long group of pictures (GOP) consisting of a large number of frames to be transferred with a low bit rate (for example, 3 to 9 Mbps) should be convertible to a short GOP consisting of a small number of frames to be transferred with a high bit rate (18 to 50 Mbps) and vice versa. In the long GOP, information is closely correlated so that the image quality is not deteriorated even if it is transferred with a low bit rate.
With reference to FIG. 1, a description is now given of a system in which frames can be edited after stream data consisting of long GOPs transmitted and received via a transmission channel is recoded into all intra-stream data having short GOPs.
Upon receiving a non-compressed original image, a coding device 1 codes it into long GOPs according to the MPEG method and transmits the stream data to a transcoder 2 via a transmission channel 11. In the transmission channel 11, stream data consisting of long GOPs, which are suitable for transmission, are transmitted.
In the transcoder 2, after the MPEG long GOP stream data supplied via the transmission channel 11 is decoded in a decoder 21, it is recoded to all intra-stream frames in a coder 22, and the coded all intra-stream data (serial data transport interface contents package (SDTI CP) stream) is output to a frame editing device 3 of an SDTI CP interface.
The stream data edited in the frame editing device 3 is supplied to a transcoder 4. In the transcoder 4, after the supplied all intra-stream data is decoded in a decoder 31, it is recoded into MPEG long GOPs in a coder 32, and the coded MPEG long GOP stream data is output to a predetermined data receiver via the transmission channel 11.
With reference to FIG. 2, a description is now given of a system for coding an input image into MPEG long GOPs with a high bit rate and for decoding the MPEG long GOPs and recoding them into low-bit-rate MPEG long GOPs.
The coding device 1 codes a non-compressed input image into high-bit-rate MPEG long GOPs and outputs the coded MPEG long GOP stream data. A transcoder 51 decodes the high-bit-rate MPEG long GOPs in a decoder 71 and then recodes them into the low-bit-rate MPEG long GOPs in a coder 72. The transcoder 51 then outputs the coded low-bit-rate MPEG long GOPs to a predetermined data receiver via the transmission channel 11.
In this manner, when repeatedly coding and decoding image information, the image quality becomes lower if coding parameters are changed every time coding is performed. To overcome this drawback, a technique for suppressing the deterioration of the image quality caused by recoding by using coding history information, which is inserted into the user data area of the picture layer of a bit stream, is disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2000-059788.
For example, in a system in which MPEG long GOPs can be converted into short GOPs, which can be edited in units of frames, recoding is performed by using coding history information. The configuration of this system is described below with reference to FIG. 3. In FIG. 3, elements corresponding to those of FIG. 1 are indicated by like reference numerals, and an explanation thereof is thus omitted.
A transcoder 101 receives MPEG long GOPs from the coding device 1 via the transmission channel 11.
An MPEG long GOP consists of three types of pictures having different characteristics, i.e., intra-coded (I) pictures, predictive-coded (P) pictures, and bidirectionally predictive-coded (B) pictures. Accordingly, video data decoded from such an MPEG long GOP also consists of I pictures, P pictures, and B pictures. Thus, when recoding the video data into the MPEG long GOPs, if the I pictures, P pictures, and B pictures of the video data are recoded with different types of pictures, the image quality may be decreased. For example, if a B picture, which is more susceptible to distortions than I pictures or P pictures, is recoded as an I picture, the preceding or upcoming pictures obtain predictions from this distorted I picture, thereby decreasing the image quality.
To prevent the deterioration of the image quality caused by recoding, upon receiving, via the transmission channel 11, stream data that was previously coded in another transcoder or a coding device, a transcoder 101 performs the following operation. After decoding the supplied MPEG long GOP stream data in a decoder 111, when recoding the decoded MPEG long GOP stream data into all intra-stream frames in a coder 112, the transcoder 101 adds parameters, such as the picture types and quantizing levels, of the coded stream supplied to the decoder 111 to the all intra-coded stream as Society of Motion Picture and Television Engineers (SMPTE) 328M history information, and supplies the all intra-coded stream to the frame editing device 3.
The stream data edited in the frame editing device 3 is supplied to a transcoder 102. The transcoder 102 decodes the all intra-stream data with history information in a decoder 121. A coder 122 recodes the all intra-stream data into long GOPs by using parameters, such as the picture types and quantizing levels, contained in the decoded history information.
There is a system in which non-compressed data is coded into MPEG long GOPs with a high bit rate and the MPEG long GOPs are decoded, and the decoded data is recoded into low-bit-rate MPEG long GOPs. In this system, the deterioration of the image quality caused by recoding is suppressed. The configuration of this system is described below with reference to FIG. 4. In FIG. 4, elements corresponding to those in FIG. 2 are designated with like reference numerals, and an explanation thereof is thus omitted.
Upon receiving an MPEG long GOP stream coded in the coding device 1, a transcoder 131 obtains required coding parameters when decoding the high-bit-rate MPEG long GOP in a decoder 141, and supplies the decoded video data and the obtained coding parameters to a coder 142. The coder 142 recodes the video data into low-bit-rate MPEG long GOPs by using the supplied coding parameters, and outputs the coded low-bit-rate MPEG long GOP stream.
When using history information and coding parameters obtained during the decoding operation so as to prevent the deterioration of the image quality, the image data of a coded stream that is edited in the frame editing device 3 becomes discontinuous, thereby making the information indicating the correlation between frames meaningless. Accordingly, when using history information and coding parameters, the editing points of the stream data must be detected, and at the editing points, recoding must be performed without using the history information and coding parameters.
The following technique is disclosed in Japanese Unexamined Patent Application Publication No. 2003-143607. In editing code sequences in units of GOPs, the image types (I, P, B) are determined from the headers of input code sequences A and B to be edited, and codes (flags) indicating editing points are inserted or rewritten based on the image types of input code sequences A and B and externally input editing information, thereby indicating the editing points of the stream data.
The following technique disclosed in Japanese Unexamined Patent Application Publication No. 2001-169278 is also available. Information indicating a count number for counting frames or fields is added to stream data. When recoding images, discontinuous points of images generated by splicing, inserting, and removing the images are detected according to whether the count number is continuous.
As described above, the image quality can be maintained by performing recoding by reusing previous coding information, such as history information or coding parameters (parameters inserted into the picture layer and the macroblock layer, such as picture types, motion vectors, and quantizing levels). However, for example, during editing, streams having a bit rate, an image frame, or a chroma format that are different from that used in the previous coding may be disposed or inserted instead of the streams used in the previous coding. In this case, parameter information used in the previous coding cannot be reused for performing recoding.
As discussed above, there is one technique in which codes (flags) indicating editing points are inserted for detecting editing points. In this case, in the entire system for sending and receiving coded streams, all editing devices should be able to insert predetermined codes (flags) indicating editing points, and all coding devices should be able to detect such codes (flags). If there is one editing device that does not have a function of inserting codes (flags) indicating editing points in the system, a coded stream may include editing points without predetermined codes (flags).
As discussed above, there is another technique in which information indicating the count number for counting frames or fields is added to stream data. With this technique, even if there is an editing device without a function of inserting codes (flags) indicating editing points in the system, the editing points can be detected from the continuity of the count number. However, since the counter cannot infinitely count the number, the count number may become continuous at an editing point. Additionally, when a plurality of cameras are used to start imaging and images are time-sequentially edited by switching the cameras, the count number becomes continuous at editing points. More specifically, for example, in broadcasting a baseball game using a plurality of cameras, if camera A at a backstop and camera B at a bleacher start imaging at the same time and if a pitcher imaged by camera A and a batter imaged by camera B are time-sequentially switched, such a switching point becomes an editing point. However, the count number contained in a frame or field becomes continuous.